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Abstract 

The £i-synthesis and ^i-analysis models recover structured signals from their undersampled measure- 
ments. The solution of the former model is often a sparse sum of dictionary atoms, and that of the 
^SJ ■ latter model often makes sparse correlations with dictionary atoms. This paper addresses the question: 

when can we trust these models to recover specific signals? We answer the question with a necessary 
and sufficient condition that guarantees the recovery to be unique and exact and that also guarantees 
the recovery is robust in presence of measurement noise. The condition is one-for-all in the sense that 
'^ I it applies to both of the ^i-synthesis and ^i-analysis models, and to both of their constrained and un- 

constrained formulations. Furthermore, a convex infinity-norm program is introduced for numerically 
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verifying the condition. The comparison with related existing conditions are included. 
Keywords: exact recovery, robust recovery, ^i-analysis, £i-synthcsis, sparse optimization, compressive 
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Tt ! 1 Introduction 
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^ ' Let X* G R" be a signal of interest. This paper studies when ii minimization can uniquely and robustly 

recover x* from its linear measurements 

b = <Px*+w, (1) 

X' 
^H ■ where $ £ j^mxn jg ^ certain matrix and w E R™ is noise. We focus on the compressive setting m < n 



The results of this paper cover the following £i minimization formulations: 

minimize H^^'^xHi, subject to $a; = b, (2a) 

X 

minimize ||$a; - b\\l + A||*^x||i, (2b) 

X 

minimize II 'I'"'"a;||i, subject to ||<i>x — 6||2 < 5, (2c) 

X 

where 6, A are positive parameters. When ^ — Id, the identify matrix, models in ([2]) are referred to as the 
£i (or more generally, £i-synthesis) models. When \1/ 7^ Id, they are referred to as the £i-analysis models 
(see [5] for a recent overview). 
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In synthesis models, the signal of interest is synthesized as x* = Dc, where D is a certain dictionary 
and c has the sparse coefficients. The analysis model recently attracts a lot of attention. It is assumed in 
[51 ll4[[T5] that the underlying signal makes sparse correlations with the columns (atoms) in a dictionary D, 
i.e., D^x* is sparse. The signal recover model is (j2b[) with ^ set to D. 

For both synthesis and analysis models, one is interested in when the recovery is successful, that is, 
the solution is unique and the solution error is proportional to the amount of noise. There are several non- 
universal (applied to specific sparse signals) and universal (applied to all sparse signals) conditions addressing 
questions in various forms for £i-synthesis minimization; examples include the non-universal dual certificate 
condition [7] and the "RIPless" property [1] , and universal conditions such as the restricted isometry principle 
[2], the null space condition [5], the spherical section property |20j . and others. Since £i-analysis minimization 
takes a more general form than ^i-synthesis minimization, some of the above conditions have been extended 
to the analysis case; recent works [HI HSl [HI HSl HSl [12 have made significant contributions. 

Regarding a specific signal x* , this paper establishes a necessary and sufficient condition that guarantees 
the unique solution of any model in ^ and also that the solutions of models (j2bp and ([^ are robust to any 
noise added to b. A method based on £,^ minimization that verifies the condition is presented. In addition, 
the proposed condition is compared to other conditions in the literature, most of which are stronger than 
ours and are thus sufficient but not necessary. Certain parts of our proofs are inspired by [TJ [HI [IHl [H] • 

Notation. We equip M" with the canonical scalar product (•, •) and Euclidean norm || • ||2. We let | • | 
return the cardinality if the input is a set or the absolute value if the input is a number. For any x € M", 
supp(a;) = {fc : 1 < fc < n,.Tfe 7^ 0} is the index set of the non-zero entries of x. sign(x) is the vector whose 
ith. entry is the sign of Xi, taking a value among +1, —1, and 0. For any p > 1, the ^p-norm of a; £ K" is 




its ^o-"norm" is ||x||o = |supp(a;)|, and its €oo-norm is ||x|joo = max{|a;i| : i ~ !,■■■ ,n}. For x G M" and 
/ C {1, 2, • • • ,n}, xi denotes the vector formed by the entries Xi of x ioi i £ I, and /"^ is the complement of /. 
Similarly, Aj is the submatrix formed by the columns of A indexed by /. A'^ is the transpose of A. We use 
Aj for the transpose of submatrix Aj, not a submatrix of A'^ . For square matrix A, Aniax(^) and X^in{A) 
denote its largest and smallest eigenvalues, respectively, Cond(A) denotes its condition number, and ||A|| 
denotes its spectral norm. The null and column spaces of A are denoted by Kei{A) and Im.{A), respectively. 
Outline. The rest of the paper is organized as follows. Section [2] states the main results of this paper. 
Section [3] reviews several related results. Section |4] discusses condition verification. Proofs for the main 
results are given in sections [5] and [6l 

2 Main condition and results 
2.1 Main condition 

Condition 1. Given x S M", index sets I = supp(vl'^a;) C {1, • • • , /} and J ~ V- satisfy 

(1) Kcr(*5') n Kcr($) = {0}; 

(2) There exists y eM} such that *y e Im(<I)^), yj = sign(*|'x), and ||y,/||oo < 1- 

Part (1) of the condition says that there does not exist any nonzero Ax satisfying both ^Js ~ *j(a;-|-Aa;) 
and $5; — $(x-|-Aa;). Otherwise, there exists a nonempty intervall — [x— aAx, x-f aAx] for some sufficiently 
small a > so that $x = $5; and ||^"^a;||i is linear for x G I; hence x cannot be the unique minimizcr. Part 



(2) states the existence of a strictly-complementary dual certificate y. To see this, let us check a part of the 
optimality conditions of pa)): G ^9j| • ||i(^''"x) — $"^/3, where vector /3 is the Lagrangian multipliers; we 
can rewrite the condition as = \l/y — $"^/3, with y € 9|| • ||i(^"^x), which translates to yi = s\gx\{'^^x) and 
||?/j||oo < 1- This y certifies the optimality of x. For solution uniqueness and/or robustness, we shall later 
show that llyjiloo < 1 strictly is needed. 

A variant of Condition [1] is given as follows, which is equivalent to Condition [5] below from [9]. 



Condition 2. Given x e R", let I = supp('I'"^a;). There exists a nonempty index set J Q I^ such that the 
index sets I, J and K ^ {I [J JY satisfy 

(1) Ker(*5') n Ker($) = {O}; 

(2) There exists j/ G M' such that "iiy e Im($^), yj = sign(*|"x), ||yj||oo < 1, and \\yK\\oo < 1- 



In Condition [21 a smaller J relaxes part (2) but gives a larger Ker(^J) and thus tightens part (1). 
Although Condition [2] allows a more flexible J than Condition [1] we shall show that they are equivalent. 

2.2 Main results 

Depending on the specific models in ([2|), we need the following assumptions: 
Assumption 1. Matrix $ has full row-rank. 

Assumption 2. A„aa;(**^) = 1- 

Assumption 3. Matrix ^P has full row-rank. 

Assumptions[T]and|3]arc standard. Assumption[2]is non-essential. We can scale a general ^ by multiplying 
it with , ^ Below we state our main results, whose proofs are given in sections [S] and [HI 

Theorem 1 (Uniqueness). Under Assumption \Tl let x be a solution to problem \2a]) or i2b\] . or under 
Assumptions\^and\^ let x be a solution to problem (flc)). The followings are equivalent: 

1) Solution X is unique; 

2) Condition 1 holds for x = x; 

3) Condition 2 holds for x = x. 

This theorem states that Conditions 1 and 2 are equivalent, and they are necessary and sufficient for a 
solution X to problem (|2a|). or to problem (|2b[) . or to problem ([2c|) to be unique. To state our next result on 
robustness, we let 

r{J) := sup II II ■ 

««EKor(*J)\{0} ll*"l|2 

Part (1) of Condition [T] ensures that < r{J) < +oo. If 4* = /, then u E Ker($j)\{0} is a sparse nonzero 
vector with maximal support J^, so r{J) is the inverse of the minimal singular value of the submatrix ^jc. 
Below we claim Condition [T] ensures the robustness of problems (|2b[) and (Pc)) to arbitrary noise in b. 

Theorem 2 (Robustness). Under Assumptions[Tl[3[ given an original signal x* E R", let I = supp(^"^x*) 
and J = I"^. For arbitrary noise w, let b = $a;* + w. If Condition\^ is met for x ^ x* and \\w\\2 < S, then 

1) For any Cq > 0, there exists constant Ci > such that every minimizer xg.x of problem \2b\) using 
parameter A = Cq5 satisfies 

\\^^{x&,x-x*)\\i<Ci5- 



2) Every minimizer xg of problem i2c\) satisfies 

Moreover, defining 

P=m-r^y, C,^riJ)^\ and c, . i±-^-d(*)ll*ll^3 



\yj\\ 



we can let 



Ci=2C3 + Co|l/3|l2 + 

C2=2C3 + 2C4||/3||2. 



(l + Co||/3||2/2)^C4 
Co 



Remark 1. From the results of Theorem\^ it is straightforward to derive £i or £2 bounds for {xs.\ — x*) 
and (xs — x*) under Assumption\^ 

Remark 2. Since Cq is free to choose, one can choose the optimal Cq — ./ ,,„,, y^^, ,, „,,;j and simplify Ci to 

Ci = 2C3 + c^WPh + ^Jcim\l + ^c4Ph < 2C3 + 2C4II/31I2 + 2, 



which becomes very similar to C2 ■ This reflects the equivalence between problems (|2bp and (|2cp in the sense 
that given \, one can find S so that they have the same solution, and vice versa. 

Remark 3. Both Ci and C2 are the sum 0/2C3 and other terms. 2C3 alone bounds the error when '^"^xs.x 
(or '^ Xs) and ^ x* have matching signs. Since C3 does not depend on yj, part (2) of Condition[l\ does 
not play any role, whereas part (1) plays the major role. When the signs of^'^xs^x (or ^'^xs) and '^^x* do 
not match, the remaining terms in Ci and C2 are involved, and they are affected part (2) of Condition [7J' in 
particular, \\yj\\ co < 1 plays a big role as C4 is inversely proportional to 1 — ||y||oo- Also, since there is no 
knowledge about the support of 'i'^xs^x, which may or may not equal to that of'^'^x*, C4 inevitably depends 
the global properties of '^ and $. In contrast, C3 only depends on the restricted property 0/$. 

3 Related works 

In the case oi 'i/ ~ Id, Condition [1] is well known in the literature for £1 (or ^i-synthesis) minimization. It 
is initially proposed in [7] as a sufficient condition for the £1 solution uniqueness. For problems (j2b[) and 
pc|. j8l [TT] present sufficient but non- necessary conditions for solutions uniqueness. Later, its necessity is 
established in [10] for model (j2bp and then in ^9\ for all models in ([2]), assuming '^ = Id or equal to an 
orthogonal basis. The solution robustness of model (j2bp is given under the same condition in [10]. Below we 
restrict our literature review to results for the ^i-analysis model. 

3.1 Previous uniqueness conditions 

Papers [9j |T4j [181 [11] cover the uniqueness of the £i-analysis model and use stronger conditions than ours. 
The following condition in |14| guarantees the solution uniqueness for problem ([2a]): 



Condition 3. Given x, let Q be a basis matrix o/Ker($), and I = supp(^"^a;). The followings are met: 
(1) ^^aQ is full column rank; 
(2)\\{Q^^i.)+Q'^-^isign{-^^x)\\o. < 1. 



Paper [TB] proposes the following condition for the solution uniqueness and robustness for problems (Pa)) 
and (j2b[) (the robustness requires the non-zero entries of "^Jx to be sufficient large compared to noise). 

Condition 4. For a given x, index sets I ~ supp(^ S) and J ^ I'^ satisfy: 

(l)Kev{^^j)r\Kev{^) = {Q}; 

(2)LetA^-'^ = C/(C/^$^$C/)-iC/^ andO.^'^^ = *+($^$^["'l-/<i)*/, where U is a basis matrix o/Ker(*5). 
Then 

/C(sign(^f x)) := min ||f7["'lsign(*f x) - u||oo < 1- 

MeKcr(iI'j) 

According to [18], Conditions [3] and |4] do not contain each other. 

The following example shows that Conditions [3] and |4] are both stronger than Conditions [1] and [2j Let 

-' 0,0,, -(:;j), -(-;.), -(-)^ 

It is straightforward to verify that Conditions [1] and [2] hold. However, Conditions [3] and |4] fail to hold. Indeed, 
we have l-^x = (10.5, 0, 0)"^ and I = {1}. Q = (1, 0, 0)"^ is a basis matrix of Ker($). Thus 




+ nT w, ,„;„„( i\,T ~\ 



|(g'*/0+Q */sign(*j.T 



/10.5 105\^ 




105 


^101' 101 j 


OO 


101 



(3) 



Hence, Condition |3] does not hold. Furthermore, U = (1, —1, —10)'^ is a basis matrix of Ker(^j), and the 
definition of fjl"'! gives us Sl^ ^ (i^, j^Y ■ Therefore, /C(sign(^f i)) = i25 > 1, so Condition H does not 
hold either. Paper [TH| also presents sufficient conditions for solution uniqueness, which are reviewed in [TH] 
and shown to be not necessary. 

3.2 Previous robustness conditions 

Turning to solution robustness, E] have studied the robustness of problems pbp and (Pc)) in the Hilbert- 
space setting. Translating to the finite dimension, the conditions in read: 

Condition 5. Given x, the following two statements hold: 

(1) Ker(*J) n Ker($) = {O}; 

(2) There exists y G d\\ ■ ||i(*^S) such that ^y G Im($^) and J == {i : |yi| < 1} 7^ 0. 

This condition is equivalent to Condition[2] Under Condition^ [9] shows the existence of constant C (not 
explicitly given) such that the solution xs.\ to (|2b|) obeys ||^^(a;5^A — a;*)||2 < CS when A is set proportional 
to the noise level S. In order to obtain an explicit formula for C, [TT] introduces the following: 

Condition 6. Let 'J = (^1"!'-'")^^$. Given x, the following two statements hold: 

(1) There exists some y G 9|| • ||i(^-'"x) such that '^y G Im($-^); 

(2) For some t G (0, 1), letting I(t) = {i : \yi\ > t}, the mapping $ := 'I'|gpan{* iG/ft)! *'^ injective. 



Under this condition, the solutions to ()2bp and (pc|) are subject to error bounds whose constants depend 
on t, $, the lower frame bound of ^, and other quantities. 

Proposition 1. Condition\^is stronger than Gondition\^ 



Proof. Let J = I{tY; then we have Hy./Hoo < t < 1 from the definition of I{t). It remains to show that 
Ker(^J)nKer(<i>) = {0}. For any x e Ker(vl/J), we have 

Since $ restricted to Spanj^; : i G /(t)} = Im((^^-^)~^'I'/(t)) is injective, we have what we need. D 

Paper [T^ provides a much stronger condition below that strengthens Condition U by dropping the 
dependence on the ^-support (see the definition of RC{I) below). 

Condition 7. Given x, index sets I = supp(^^5:) and J ^ I'^ satisfy: 
(l)KeT{^^j)f]Ker{^) = {0}; 
(2) Letting fi^'^i he given as in Condition^ 

RC{T) := max min \\D,^''^p - u\\oo < I- 

Under this condition, a nice error bound and a certain kind of "weak" sign consistency (between 'i'^xs.x 
and ^P-^x*) are given provided that problem (|2bp is solved with the parameter A = 2(i-rc^(i)) ^'-"" some p > 1, 
where cj — ||vl'+$^($y4['^l$^ — /(i)||2.oo- When 1 — RC{I) gets close to 0, this A can become too large than 
it should be. 

4 Verifying the conditions 

In this section, we present a method to verify Condition [TJ Our method includes two steps: 

(Step 1:) Let $ = UHV'^ be the singular value decomposition of $. Assume V = [vi, • • • , v„]. Since $ 

has full row-rank, we have Ker($) = Span{wm+i, • • • , w„} and Q = [wm+i, • • • , Wn] as a basis of Ker($). We 

verify that ^jQ has full row-rank, ensuring part (1) of Condition [TJ 

(Step 2:) Let ui — — Q^'I'/sign(\I'j x) and A = Q^'^ j. Solve the convex problem 

minimize Halloo, subject to Au = ui. (4) 

If the optimal objective of ([4]) is strictly less than 1, then part (2) of Condition [T] holds. In fact, we have: 

Proposition 2. Part (2) of Condition]^ holds if and only if Q) has an optimal objective < 1. 

Proof. Let u be a minimizer of ([4]). Assume |lw||oo < 1- We consider the vector y composed by yj — sign(^Jj:) 
and yj = u. To show part (2) of Condition [U it suffices to prove ^y £ Im(<f>-^), or equivalently, Q^'^y ~ 0. 
Indeed, 

Q^^y = Q'^^jyj + Q'^-^iyi = Q^'^ju + Q^^m = 0. 
The converse is obvious. D 

Convex program ^ is similar in form to one in jl8) though they are used to verify different conditions. 

5 Proof of Theorem 1 

We establish Theorem [T] in two steps. Our first step proves the theorem for problem (Pa|) only. The second 
step proves Theorem [1] for problems (pb| and (Pc| . 

6 



5.1 Proof of Theorem [T] for problem (1 2 ah 

The equivalence of the three statements is shown in the following orders: 3) ==^ 1) =^ 2) =^ 3). 

3) => 1). Consider any perturbation x + h where h e Ker($)\{0}. Take a subgradient g e d\\ ■ J|i(^^.t) 
obeying gj = sign(^Jx) ~ yj, qk = Vk, and \\gj\\ < 1 such that {gj,'i''^h) = ||^J/i||i. Then, 

\\^^ix + h)\\i>\\^^x\\i + {^g,h) (5a) 

= ||*^;r||i + (*g-*y,/i) (5b) 

= \\^^x\\i + {g-y,^^h) (5c) 

= ||*^x||i + (.gj-yj,*J/i) (5d) 

>||vl/^x||i + ||vl/>||i(l-j|y,;||^), (5e) 

where (|5b|) follows from ^Py e Im($^) = Ker($)^ and h G Kcr(<i>), (j5d|) follows from the setting of g, and 
((5e| is an application of the inequality (x,y) < ||a;|ji||y||oo and {gj,'^^h) = ||^j/i||i. Since h G Kcr('I>)\{0} 
and Ker(vI/J)f|Ker($) = {0}, we have |i*j/i||i > 0. Together with the condition ||yj||oo < 1, we have 



ll'I' (x + /i)||i > 11^ .t||i for every h G Ker($)\{0} which implies that x is the unique minimizer of (Pa|) . 

1) =^ 2). For every h G Ker($)\{0}, we have $(.t + th) = $a; and can find t small enough around 
such that sign(*I'|'(a; + th)) = sign(^j2;). Since x is the unique solution, for small and nonzero t we have 

||*^(x)||i<||*^(x + i/i)||i = ||*f(i + t/i)||i + ||*f.(.T + i/i)||i (6a) 

= (*f (i + th), sign(^f (x + th))) + \\t^]M\\ 1 (6b) 

= (*ff + i*f/i,sign(*fa;-)) + ||f*fe/ii|i (6c) 

= (*f X, sign(^f £)) + t{^]K sign(*f £)) + \\t^]M\\ i (6d) 

= ||*^(x)||i+t(*f/i,sign(*fx)) + i|t*f./i||i. (6e) 

Therefore, for any h G Kcr($)\{0}, we have 

(*f/i,sign(*f£))<||*j:/i||i. (7) 

If the condition Ker(^|L ) H Ker($) = {0} does not hold, we can choose a nonzero vector h G Ker(^ fc ) {~\ Ker($) . 
We also have -h G Ker(*f<,) f] Ker($). Then we have (*f /i,sign(*|'x)) < and -(^J^/i, sign(*fx)) < 0, 
which is a contradiction. 

It remains to show the existence of y in item (2) of Condition [TJ This part is in spirit of the methods 
in papers [TO] and [TO], which are based on linear programming strong duality. We take y with restrictions 
iji = sign(^-fx) and y/c ~ 0. If such y satisfies ^y G Im($-^), then the existence has been shown. If 
^y ^ Im($-^) = Ker($)-'-, then we shall construct a new vector to satisfy part (2) of Condition [1] Let Q 
be a basis matrix of Ker($). We have that a := Q^'^y must be a nonzero vector. Consider the following 
problem 

minimize ||2;||oo subject to Q^'^z = —a and zj = 0. (8) 

For any minimizer z of problem ([8]), we have ^^(y + z) G Kcr($)^ = Im($"^) and (y + z)i = y/ = sign(\I'Ja;). 
Thus, we shall show that the objective of problem ([5]) is strictly less than 1. To this end, we rewrite problem 
([5]) in an equivalent form as: 

minimize II z/c I loo subject to Q^^/^z/c = —a, (9) 

z 

whose Lagrange dual problem is 

maximize(p,a) subject to IJ'I'fcQpIJi < 1. (10) 

p 



Note that Qpe Kcr($) and \{p,a)\ = \{p,Q'^-^y)\ = |(p, (3'^*/sign(*J'x))| = \{'i'J Qp, sign{^ J x))\. By using 
([7]) , for any p we have 

i \{^jQp,sign{^Jx))\=0, HQp^O; 

'^^'"^' \ |(*fQp,sign(*fi))|<|l*|;Qp|li<l, otherwise. ^ ' 

Hence, problem pop is feasible, and its objective value is strictly less than 1. By the linear programming 
strong duality property, problems ([8]) and ([9]) also have solutions, and their the objective value is strictly 
less than 1, too. This completes the proof. 

2) =^ 3). Let J = r and K = 0; then Condition 2 follows. 

The proof of 3) =^ 1) is a standard technique in compressed sensing community. 

5.2 Proof of Theorem [1] for problems ( I2bj) and (I2cl) 

Lemma 1. Let 7 > 0. Ifj\\^x— 6|||+ ||^'^a;||i is constant on a convex set ^, then both $x — 6 and \\'^'^x\\i 
are constant on Q. 

Proof. It suffices to prove the case where the convex set has more than one point. Suppose xi and X2 are 
arbitrary two different points in il. Consider the line segment L connecting xi and X2. By the convexity of 
set ri, we know L C fl. Thus, c = 7||$a; — b\\2 + j|^"^a;||i is a constant on L. If $.ti — & 7^ $2:2 — b, then for 
any < a < 1, we have 

7|l$(aa;i + (1 - a)x2) - b\\l + \\-^'^{axi + (1 - a)a::2)|li (12a) 

= -f\\a{^xi - 5) + (1 - a)($a;2 - b)\\l + ||a(*^xi) + (1 - a)(*^a;2)|li (12b) 

< a(7||$a;i -b\\l + \\'^'^xi\\i) + (1 - a){j\\^X2 -b\\l + ||*'^2;2||i) (12c) 

= ac+{l- a)c = c, (12d) 

where the strict inequality follows from the strict convexity of 7II • II2 and the convexity of ||^"^a;||i. This 
means that the points axi + (1 — a)x2 on L attain a lower value than c, which is a contradiction. Therefore, 
we have ^xi — & = ^X2 — b, from which it is easy to see ||*^xij|i = ||\I'-^:r2||i. D 



We let Xx and Ys denote the sets of solutions to problems (j2b| and (f2c|) . respectively; moreover, we 
assume that these two sets are nonempty. Then, from Lemma [Ij we have the following result. 

Corollary 1. In problem Ii2b]) . ^x — b and \\'9'^x\\i are constant on Xx; in problem i2c\) . $a; — 6 and \\'^^x\\i 
are constant on Yg. 

Proof. Since ||$a; — fejjj + AH^P-^xHi is constant over Xx, the result follows directly from Lemma[T]for prob- 
lem (|2bp . For problem (pc|) . if G Yg, then we have Yg = {0} because of the full row-rankness of ^. The 
result holds trivially. Suppose ^ Yg. Since the optimal objective ||\E'"^a;||i is constant for all x £ Yg, we 
have to show that \\^x — 6||| = S for all x G Yg. If there exist a nonzero x &Yg such that \\^x — b\\l < S, we 
can find a non-empty ball B centered at x with a sufficiently small radius p > such that \\^x — 6|J2 < 6 for 
all X e B. Let a = min{2TTfTn-, 5} e (0, 1). We have (1 - a)x e B and ||(1 - a)*^x||i < ||*^x||i, which is a 
contradiction. D 

Proof of TheoremUl for problems l2Bjl and ^2djl . This proof exploits Corollary[T] Since the results of Corol- 
lary [T] are identical for problems ()2bp and (|2cl). we present the proof for problem (j2b|) only. 



By assumption, X\ is nonempty so we pick x e X\. Let h* = $x, which is independent of the choice of 
X according to CoroUary[T] We introduce the following problem 

minimize ||^"^a:;||i, subject to $a; — b* , (13) 

X 

and let X* denote its solution set. 

Now, we show that X\ ~ X*. Since $x = ^x and ||\E'^a::||i = ||'I'-^i:||i for all x E X\ and conversely 
any x obeying ^x = ^x and ||\E''^x||i = ||'I'-^i;||i belongs to X\, it suffices to show that ||\E''^3;||i = j|^^a;||i 
for any x G X*. Assuming this docs not hold, then since problem ([T^ has i as a feasible solution and has 
a finite objective, we have a nonempty X* and there exists x E X* satisfying ||\l/-^a;||i < ||\l/"^i;||i. But, 
W^x — 6II2 = II&* — &II2 = W^i — 6JI2 and j|'I'-^a;||i < ||^-^i;||i mean that a; is a strictly better solution to 
problem (j2bp than x, contradicting the assumption x G X\. 

Since X\ = X*, x is the unique solution to problem (|2b[) if and only if it is the unique solution to 
problem p^ . Since problem P^ is in the same form of problem (Pa]) , applying the part of Theorem [T] for 
problem (|2ap , which is already proved, we conclude that x is the unique solution to problem (j2bp if and only 
if Condition [1] or ^ holds. D 

6 Proof of Theorem 2 

Lemma 2. Assume that vectors x and y satisfy Condition\^ Let I = supp(^"^a;) and J ~ I'''. We have 

\\-^^x-^^x\\i<C3\\<^{x-x)\\2 + C4dy{x,x), Vz, (14) 

where dy{x,x) := |j^"^x||i — ||4'"^a;j|i — {'^y,x — x) is the Bregman distance of function ||^"^- jji, the absolute 
constants C^^Ci are given in Theorem\^ 

Proof. This proof is divided into two parts. They are partially inspired by [9]. 

1. this part shows that for any u G Kcr(^j), 

W^^x - vl/^xlli < ( 1 + ^=^^1= ) ||vp^(^ _ ^)||^ + C3||$(x - x)h. (15) 

V V-^mi„(**^)/ 

2. this part shows that 

/(x):=niin{||*^(x-7/)|ji: 7. e Kcr(*5)} < (1 - ||yj|U)-'d,(x, x). (16) 

Using the definition of C4, combining (|15p and ([T5)) gives P^ . 

Part 1. Let u G Ker('I'j). By the triangle inequality of norms, we get 

||*^a;-*^S||i < \\^'^{x-u)\\i + ||*^(w-S)|li. (17) 

Since x € Kcr(^j), we have u — x E Kcr(^j) and thus J|m — a;||2 < r{J)\\^{u — x)\\2, where r{J) < +cx) 
follows from part (1) of Condition[TJ Using the fact that supp(^^(M — x)) = /, we derive that 



I* (w - x)\\i < Vl^lll* (^ - ^)l|2 (18a) 

<^AI\\\u-x\\2 (18b) 

<^/\^\riJ)\\^u-x)\\2 (18c) 

= C3|l$(w-x-)||2, (18d) 
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where we have used the assumption Amax(^^ ) = 1 and the definition C3 = r(J)-\/|/|. Furthermore, 

Mu - x)|J2 < mx -u)\\2 + Mx - x)\\2 (19a) 

<||$||||x-7/||2 + ||$(x-x)||2 (19b) 

^ il^llll'^^(^-")lli , 11^^ -Ml no^A 

V^min(1'*^) 

Therefore, we get ([IS]) after combining dTT]), (UHl), and (fT9| . 

Part £ Since (^y, S) = j|4'-^S||i implies dy{x,x) = \\'^^x\\i — {'^y,x), it is equivalent to proving 

/(a:) < (1 - i|j/,/i|ocO"'(ll*^:^ili - (*y,^))- (20) 

Since u € Ker(\I'j) is equivalent to 'i^u = 0, the Lagrangian of the minimization problem in ()16p is 

L{u, v) = 11*^(2- ~ u)\\i + {v, ^Ju) = ||*^(x - u)|li + (*,7l>, u - x) + (*jw, x). (21) 

Then, /(a:) = min„ max„ L(u,v). Following the minimax theorem, we derive that 

f{x) = maxmini(u, w) =maxmin{||^^(a; — u)||i + {^jv,u — x) + (^jw, a;)} (22a) 



V U V u 

maxmin{||^'^(x — w)||i + (w, w — .t) + (ui, x) : w G Im(^,7)} (22b) 



= max{(u;,a;) : w G a||*^ • ||i(0) nlm(*,/)} (22c) 

W 

= max{(c*2/ + w,a;) : w £ 5||*^- ||i(0) nlm(*,7)} - {c^y,x), Vc> (22d) 

=cmax{(*j/ + w,a;) : w e c^^ail*^- ||i(0)nlm(*j)}-c(*y,a;), Vc> 0. (22e) 

It; 

Let 

c = (l-||y,7||oo)~' 

and Zj — {z ^ M.^ : zj = 0}. Since |l2/j||oo < 1 from part (2) of Condition[Tl we have c < +00 and get 

(2/ + c-^9||-!li(0)nz,)ca|l.|li(0), (23) 

from which we conclude 

(*2/ + c-ia||*^ -111(0)0^*7)) caii*^ -111(0). (24) 

Hence, for any w € c"^^!!^?^ - ||i(0) n Im(*j), it holds \l/y + w C d\\'^^ ■ ||i(0), which by the convexity of 
ll'I'^ - 111 implies (vpy + w, x) < |1vI/^.t||i. Therefore, f{x) < c{\\^'^x\\i - {^y,x)). D 

Lemmas, (fl^, Theorem 3; fW^, Lemma 3.5) Suppose that x* e M" is a fixed vector obeying swpp{'^'^x*) =1 
and that there are vectors satisfying y G d\\ ■ IKvf^x*) and *j/ ~ $^/3. Then for every S > and every data 
vector b satisfying ||$x* — &II2 < ^7 the following two statements hold: 

1 ) Every minimizer xs^x of problem I2B\) satisfies d,,(xx\ . x*) < ^ I | pII^/ > and\\^xs,\~b\\2 < S + X\\13\\2; 

2)Every minimizer xg of problem Il2c\) satisfies dy{xs,x*) < 2(5||/3|J2. 

From "^y = (f>^/^ and the fuU-rankness of $, we have /3 = {^^'^)~^'iy. 
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Proof of Theorem [H Firstly, we derive that 

\\^ixs,x - x*)\\i < C^mxs^x - ^Ih + Cidy{xs,x,x*) (25a) 

< C3\\^xs,x -bh + CsW'^x* -b\\2 + Cidy{xs,x,x*) (25b) 

< C,i6 + AII/3II2) + C,6 + c,il±3M^^ (25c) 

A 

where the first and the third inequahties follow from Lemmas [5] and |31 respectively. Substituting X = CqS 
and collecting like terms in (|25cl) , we obtain the first part of Theorem [2] The second part can be proved in 
the same way. D 
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